feat: Update OpenAI runners to implement Runner protocol returning RunnerResult by jsonbailey · Pull Request #149 · launchdarkly/python-server-sdk-ai

jsonbailey · 2026-04-28T23:13:09Z

Summary

`OpenAIModelRunner` and `OpenAIAgentRunner` now formally inherit from the `Runner` protocol class.
`OpenAIModelRunner.run()` implements the unified `Runner` protocol; returns `RunnerResult` with `content`, `metrics`, `raw`, and `parsed` fields. Structured output is supported via the `output_type` parameter.
`OpenAIAgentRunner.run()` updated to return `RunnerResult`; populates `tool_calls` in `LDAIMetrics` from observed openai-agents `ToolCallItem`s.
Legacy `invoke_model()` and `invoke_structured_model()` removed — the deprecated type definitions (`ModelRunner`, `ModelResponse`, etc.) are kept in `server-ai` for provider CI compatibility until a cleanup PR removes them.

Stacking

Stacked on top of `jb/aic-2388/managed-result` (PR #148).

Test plan

`make test-openai` — all OpenAI provider tests pass
`make test` — all tests across server-ai, langchain, and openai packages pass

🤖 Generated with Claude Code

Note

Medium Risk
Medium risk because it changes the provider runner interface and result shape (e.g., run signatures and RunnerResult fields), which can break downstream integrations even though behavior is largely preserved.

Overview
Updates the OpenAI provider runners to the unified Runner protocol, replacing legacy AgentRunner/ModelRunner return types with RunnerResult (content, raw, optional parsed).

OpenAIModelRunner.run now accepts either a prompt string or LDMessage list and supports structured output via output_type (JSON schema), parsing JSON into RunnerResult.parsed when requested. OpenAIAgentRunner.run now returns RunnerResult and records observed tool invocations in LDAIMetrics.tool_calls by extracting tool call items from openai-agents run results.

Tests are updated to assert the new run() behavior and RunnerResult fields for both chat completions and structured output paths, and to reflect the agent runner’s new result shape.

^{Reviewed by Cursor Bugbot for commit 2878bda. Bugbot is set up for automated code reviews on this repo. Configure here.}

jsonbailey · 2026-04-29T13:28:44Z



-class OpenAIAgentRunner(AgentRunner):
+class OpenAIAgentRunner:


we should add the Runner protocol here vs stating it in the doc below

jsonbailey · 2026-04-29T13:30:55Z

@@ -14,10 +14,15 @@

 class OpenAIModelRunner(ModelRunner):


This should be switched to the Runner protocol.

cursor · 2026-04-29T16:38:14Z

        self._model_name = model_name
        self._parameters = parameters

-    async def invoke_model(self, messages: List[LDMessage]) -> ModelResponse:


Missing deprecated adapters breaks backward compatibility with callers

Medium Severity

The PR description states "Legacy invoke_model() and invoke_structured_model() retained as deprecated adapters that delegate to run() for backward compatibility," but these methods are completely absent from OpenAIModelRunner. The SDK's ManagedModel.invoke() calls self._model_runner.invoke_model() directly without an isinstance check, and Judge.evaluate() calls self._model_runner.invoke_structured_model(). If an OpenAIModelRunner instance is used through either of these existing code paths, it will raise AttributeError at runtime.

^{Reviewed by Cursor Bugbot for commit efeea93. Configure here.}

…nnerResult - OpenAIModelRunner.run() implements the unified Runner protocol; returns RunnerResult with content, metrics (LDAIMetrics), raw, and parsed fields. Structured output is supported via the output_type parameter. - OpenAIAgentRunner.run() updated to return RunnerResult; populates tool_calls in LDAIMetrics from observed openai-agents ToolCallItems. - Legacy invoke_model() and invoke_structured_model() retained as deprecated adapters that delegate to run() and wrap results into ModelResponse / StructuredResponse for backward compatibility. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…nner Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 2878bda. Configure here.}

cursor · 2026-04-30T14:49:27Z

+            tool_calls = [
+                tool_name
+                for _agent_name, tool_name in get_tool_calls_from_run_items(result.new_items)
+            ]


Agent runner reports function names instead of config keys

Medium Severity

The tool_calls list in LDAIMetrics is populated with raw runtime names from get_tool_calls_from_run_items (which returns Python __name__ values for custom function tools), rather than the LD config key names. The graph runner correctly builds a _tool_name_map (mapping fn.__name__ → config key) and uses it when tracking tool calls. The agent runner lacks this mapping, so it reports e.g. my_weather_function instead of the config name get-weather. This is inconsistent with the graph runner and with the tracker's documented semantics ("List of tool keys that were invoked").

^{Reviewed by Cursor Bugbot for commit 2878bda. Configure here.}

This was referenced Apr 28, 2026

feat: Update LangChain runners to implement Runner protocol returning RunnerResult #150

Open

feat: Add evaluations support to ManagedAgent.run() #153

Draft

feat: Graph tracking refactor — ManagedAgentGraph drives tracking for new runner shape #154

Draft

jsonbailey force-pushed the jb/aic-2388/managed-result branch from bd4cd68 to 45441da Compare April 29, 2026 13:14

jsonbailey force-pushed the jb/aic-2388/openai-runner-protocol branch from f1845b4 to 94f09ee Compare April 29, 2026 13:14

jsonbailey force-pushed the jb/aic-2388/managed-result branch from 45441da to 27bcfc0 Compare April 29, 2026 13:18

jsonbailey force-pushed the jb/aic-2388/openai-runner-protocol branch from 94f09ee to 8463109 Compare April 29, 2026 13:19

jsonbailey force-pushed the jb/aic-2388/managed-result branch from 27bcfc0 to ff47ec2 Compare April 29, 2026 13:22

jsonbailey force-pushed the jb/aic-2388/openai-runner-protocol branch 2 times, most recently from eacddee to 842e4e6 Compare April 29, 2026 13:46

jsonbailey force-pushed the jb/aic-2388/managed-result branch from 2ea3384 to 369242d Compare April 29, 2026 13:56

jsonbailey force-pushed the jb/aic-2388/openai-runner-protocol branch from 842e4e6 to 4c95357 Compare April 29, 2026 13:56

jsonbailey force-pushed the jb/aic-2388/managed-result branch from 369242d to b8d3fad Compare April 29, 2026 14:37

jsonbailey force-pushed the jb/aic-2388/openai-runner-protocol branch from 4c95357 to 3f6882c Compare April 29, 2026 14:38

jsonbailey force-pushed the jb/aic-2388/managed-result branch from b8d3fad to 4e28ae6 Compare April 29, 2026 16:31

jsonbailey force-pushed the jb/aic-2388/openai-runner-protocol branch from 3f6882c to efeea93 Compare April 29, 2026 16:31

jsonbailey commented Apr 29, 2026

View reviewed changes

jsonbailey marked this pull request as ready for review April 29, 2026 16:31

jsonbailey requested a review from a team as a code owner April 29, 2026 16:31

cursor Bot reviewed Apr 29, 2026

View reviewed changes

jsonbailey force-pushed the jb/aic-2388/openai-runner-protocol branch 3 times, most recently from 330acf1 to cd983aa Compare April 29, 2026 17:13

jsonbailey force-pushed the jb/aic-2388/managed-result branch from 7352a34 to adfd9f0 Compare April 29, 2026 17:14

jsonbailey force-pushed the jb/aic-2388/openai-runner-protocol branch from cd983aa to 184be64 Compare April 29, 2026 17:14

jsonbailey force-pushed the jb/aic-2388/managed-result branch from adfd9f0 to b4d15df Compare April 29, 2026 17:17

jsonbailey force-pushed the jb/aic-2388/openai-runner-protocol branch 4 times, most recently from 3556ba4 to b5f15b7 Compare April 29, 2026 21:07

jsonbailey force-pushed the jb/aic-2388/openai-runner-protocol branch 4 times, most recently from 9254ccc to d113d46 Compare April 30, 2026 14:20

jsonbailey and others added 2 commits April 30, 2026 09:42

refactor: OpenAIModelRunner and OpenAIAgentRunner formally inherit Ru…

2878bda

…nner Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

jsonbailey force-pushed the jb/aic-2388/openai-runner-protocol branch from d113d46 to 2878bda Compare April 30, 2026 14:43

cursor Bot reviewed Apr 30, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Update OpenAI runners to implement Runner protocol returning RunnerResult#149

feat: Update OpenAI runners to implement Runner protocol returning RunnerResult#149
jsonbailey wants to merge 2 commits intojb/aic-2388/managed-resultfrom
jb/aic-2388/openai-runner-protocol

jsonbailey commented Apr 28, 2026 •

edited by cursor Bot

Loading

Uh oh!

jsonbailey Apr 29, 2026

Uh oh!

jsonbailey Apr 29, 2026

Uh oh!

Uh oh!

cursor Bot Apr 29, 2026

Uh oh!

cursor Bot left a comment

Uh oh!

cursor Bot Apr 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant



		class OpenAIAgentRunner(AgentRunner):
		class OpenAIAgentRunner:

Conversation

jsonbailey commented Apr 28, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Stacking

Test plan

Uh oh!

jsonbailey Apr 29, 2026

Choose a reason for hiding this comment

Uh oh!

jsonbailey Apr 29, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cursor Bot Apr 29, 2026

Choose a reason for hiding this comment

Missing deprecated adapters breaks backward compatibility with callers

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor Bot Apr 30, 2026

Choose a reason for hiding this comment

Agent runner reports function names instead of config keys

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

jsonbailey commented Apr 28, 2026 •

edited by cursor Bot

Loading